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Unconstrained optimization problems, such as energy minimization, can be 
solved using the conjugate gradient method. For its major characteristic, the 
optimal formula conjugate encompasses all conjugate gradient algorithms. In 
conjugate gradient approaches, the formula conjugate is typically the focus 
point and it's playing a very important role for conjugate gradient 
approaches. To offer the essential descent criteria in this work, we devised a 
novel formula based on the second order Taylor which have the descent 
property too. Our research focused on our suggested method's-convergence 
property with Wolfe condition is established and numerical performance. 
Comparison to FR-method, the new algorithem shows significant 
improvement in numerical results. 
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1. INTRODUCTION 


The main goal of this study is to present an efficient optimization approach for minimizing a 
function's objective function f:R” > R, see [1]. The following equation gives the usual form of iterations 
for discovering the extreme points of the function: 


Xk+1 = Xg + Ady 


() 


certainly, a, and dẹ are the most significant aspects of an optimization model (1), and they determine the 
effectiveness of a technique. Hassan [2] has further information. The length of a step can be determined by: 


Ok i fee (2) 
typically, the Wolfe line search criteria [3], [4] are used to determine the step length a, in (1): 

fe + ade) < f(x) + Sardi dk (3) 

I(x; + Oh) = ode i (4) 


where 0 < 6 < ø < 1. The vector direction is computed as follows in the general conjugate gradient method: 


dk+1 = —Skr1 + Bed, 


(5) 
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notice that (4) is a generic update formula for the conjugate gradient parameter-based search direction 
computation. The key point in various literatures (refer to [5]-[8]) is to pick the scalar g, which leads to a 
variety of conjugate gradient techniques depending on the fẹ. Dai et al. [1] and Fletcher and Reeves [9] 
devised the conjugate gradient approach, which goes like this: 


FR _ Ideal? DR _ llgk+1ll? 
Bao = Gd. °? "K df yk (6) 


denote Y to be the gradient change. Many conjugate gradient methods have been created, and numerous 
publications have been written on the subject, addressing both computational and theoretical aspects. Iiduka 


and Narushima [10], x is calculated as: 


HY esa ll? 
= 7 
k 2/ak(fk-fk+1) ) 


as a result, the parameter's value has changed f;, in [2] is as (8). 


B Wie ll? 
= 8 
Pk FS 1) / Oe Gh ty /2 (8) 


accelerated gradient descent techniques were classified as a subclass of gradient descent methods during the 
creation of gradient models [11]. Scalar 6, may be chosen in a variety of ways, and a good choice leads to 
improved convergence. These are descent-direction approaches as well. 

In this study, we describe a conjugate gradient technique based on the conjugacy criterion. Our 
recommended strategy ensures enough descent using any line search. We can establish global convergence of 
our suggested approach under correct conditions if the line search fits the Wolfe criteria. The numbers 
suggest that the technique we propose is feasible. 


2. DERIVING NEW COEFFICIENT CONJUGATE 

The conjugacy condition is well recognized to play a crucial role in convergence analysis and 
numerical computation. The conjugacy requirement is used in this new strategy. The conjugate gradient 
approach is such that the conjugacy requirement holds in the situation when f is quadratic model and V is 
determined using the precise line search, namely: 


ditd = 0 (9) 


as a result of adding (5) to the previous equation: 


= S41 98k 
Be os (10) 


the new conjugate gradient formula is derived using the second order Taylor formula. Allow me to explain: 
1 
F(x) = f ear) + Shear & — Xr41) + 7 (x = xp) COG) xe) (11) 
the f (x) has the following gradient as: 


Gita = Se + Q(Xk+1)Sk (12) 


using (3) with exact line search in (11), we get: 


SEQ (X41) Se = eat — fe) + 3/2 ardi dk (13) 


To write (13), we have (14). 


AQ (Xe )Se = feta — fk)/ak + 3/2 9k Sk (14) 


putting (15) in (10), we obtain: 
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Be = Seta Vk (15) 
KT Fes —Fid/ Me +3/26. Sk 


we obtain the following results when we use exact line search. 


B > lidk+1ll? (16) 
k Fes —Fid/ te +3/26 Sk 


As a result of our novel formula, the so-called new method. Using this way, we will obtain the new 
algorithm which generates different formulae by using the second order Taylor formula and conjugacy 
condition. However, we may now present a new algorithm as follows, employing our suggested new 
parameter (16) as: 

Step 1: Let x, E R” and d, = —gj. Put k = 1. 

Step 2: If ||gx+1|] < 107°, then stop. 

Step 3: Estimate a, and which of the three fulfill (3-4) and variables should be updated x,41 = Xk + ayy. 
Step 4: Calculate the value of 6; as defined in (16). 

Step 5: Put k =k + 1 and go to 5. 

Theorem (2.2): for the new approach, we have the following in the form of (1) and (5), with the new 
parameter given by (16). 


dk+19k+1 < 0 and dk41Jk+1 = Ferdi dk (17) 


Proof: from evident that d} = —g; satisfies d? g, < 0. Let that d? gẹ < 0 hold for any k. Straight from (5) 
and (16) we obtained that: 


dk+1dk+1 = —Ghsr Siar + LrdkSk+i = —Lr(fk+1 — fk)/ar + 3/2 SS) + BM Sear (18) 


using (12) in above equation, we get: 


Chea Sess = brd Sk (19) 


since d? dẹ < 0, then we get: 
dk+19k+1 < 0 (20) 


so the proof is completed. 


3. CONVERGENCE ANALYSIS 

Each method in the numerical optimization methods has different convergence properties and 
needed some assumptions. Our recommended strategy ensured enough descent using any line search. In the 
study of innovative conjugate gradient procedure’ global convergence, the assumptions listed below are 
commonly utilized. Assumption A: 
— The bounded level set Q = {u/f(u) < f (uo))}. 
— The function f is smooth and its gradient is Lipschitz continuous in some neighborhood Q of Lo, 

i.e.,;where L > 0 constant, such that: 


lIs) — SII < Llv — zll, vv, z € Q (21) 
for more details see [12]. In order to prove the convergence property of our new algorithm and some 
algorithem in optimization, we use the proof idea of a researcher Zoutendijk. The Zoutendijk condition is a 
lemma that is widely used to check the conjugate gradient technique's global convergence. Zoutendijk was 


the first to say it in [13]. 


Lemma (3.1): assume that Assumption A is true. The sequence {x} produced by any iteration algorithm 
thus satisfies: 
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o (dk d)? 
esr Jad? S G2) 


Theorem (3.2): the sequence {x;,} created by the new approach meets the following conditions: 
lim inf [[g,|| = 0 (23) 


Proof: assume that (23) is false by induction. The (8) is changed to dk}1 + g,,, = pdk and when both 
sides are squared, we get: 


2 
ldel? + ldall + 2Sa = Ell? (24) 


putting (24), results in: 


afad 2 
laesa l? = E lel? = 2de Teas = Ug l (25) 


taking (25) and dividing it by Gado): we have: 


Wes all? Well? Metall? = (26) 
(dhaiSkt1)? (kI? — (har daa)® — dear Seta 


we may deduce the following from the equation: 


lldk+1ll2 III? ees ll 1 1 III? 1 
= = <- - | MM + < + 27 
(th+iSk+1)? ~ (dP gi)” [Bs | Wall T (Eg) Weal? 27) 
Hence, 
Udall? k4+1 
2 
(ha 1Sk+1)? — < Xizi TE (28) 


let there exists c4 > 0 such that IIs, = cı forall k E n , then: 


2 
lde+all a a (29) 
(dh419k+1) ci 


using the assumption and (29) as a guide, we can see that: 


œo Sede)? _ 
Diet Mera lig = (30) 


we know that jim inf |ldx|| = 0 holds because of Lemma 1. 


4. NUMERICAL RESULTS 

The results of numerical experiments are presented in this section. The FR-Algorithm was compared 
to the new conjugate gradient algorithm. In Fortran, both algorithms are written. Andrei [14] is where you'll 
find the test problems. The following papers and references are available to interested readers (Ref. [15]- 
[19]). We chose (15) unconstrained optimization problems in extended, and we investigated numerical 
experiments with the amount of variables n=100;1,000 for each test function. As a termination condition, we 
apply the inequality Ig (k+1) I<{10]^-6). Many papers have proposed this method for optimization 
problems also [20]-[26]. The Wolfe condition test results were collected and reported in Table 1, where each 
column has the following meanings; 6=0.0001 and o=0.5. The problem's name is problem, the problem's size 
is dim, the number of iterations (NOD, and the number of function evaluations are all terms used to describe 
the problem (NOF). In terms of iterations and function evaluations, Table 1 indicates how many difficulties 
these methods have resolved. 
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Table 1 compares the new methods to the Fletcher and Reeves [1] convex optimization technique; as 


demonstrated in Table 2. The new algorithm saves NI, NR, and NF overall when compared to the 
conventional Fletcher and Reeves (FR) methodology. Particularly for our collection of test issues, which 
showed the performance improvement of the new algorithm. 


5. 


Table 2. The performance percentage for the new CG algorithm compared with FR method 


Table 1. The FR and new methods numerical results 


FR algorithm New „algorithm 
RNg : NU NR NFA MN _ NR NF 
Trigonometric 100 19 12 35 18 11 33 
1000 38 23 65 37 23 65 
Extended Rosenbrock 100 47 18 93 40 23 88 
1000 78 45 131 38 21 83 
Extended Beale 100 32 15 52 16 9 30 
1000 22 10 42 13 7 25 
Generelized Tridiegonal 1 100 25 11 43 23 8 44 
1000 46 28 741 40 23 528 
Extended Tridiagonal 1 100 32 13 64 16 7 33 
1000 77 46 169 15 6 30 
Generalized Tridiagonal2 100 37 8 67 40 15 59 
1000 73 27 115 49 19 77 
Extended Himmelblau 100 12 5 25 10 6 19 
1000 14 6 29 10 6 19 
Extended PSC1 100 15 9 31 8 6 17 
1000 8 6 17 7 5 15 
Q. Diagonal Perturbed 100 124 41 231 48 9 87 
1000 445 196 711 197 46 352 
Extended Wood 100 71 35 110 26 11 50 
1000 47 15 84 34 12 65 
Extended Hiebart 100 101 40 217 83 47 190 
1000 101 40 214 86 49 194 
ARWHEAD (CUTE) 100 9 4 18 8 5 16 
1000 12 =) 82 8 5 16 
NONDIA (CUTE) 100 13 7 25 11 6 22 
1000 15 7 29 9 6 19 
DIXMAANE (CUTE) 100 121 65 218 89 34 143 
1000 345 169 634 252 86 400 
Partial Perturbed Q 100 74 21 123 79 20 120 
1000 370 88 616 251 66 436 
Total 2423 1017 5031 1561 597 3275 


NF NR NI 
FR method 100% 100% 100% 
New method 64.42% 58.70% 65.09% 


CONCLUSION 


In this study, we offer a novel nonlinear CG-algorithm based on the strictly convex quadratic 


function described by (10) that is globally convergent, functional, and fulfills the descent property under 
certain assumptions. The novel types presented in this research are successful, according to computer studies. 
The new algorithm was sufficient and effective. 
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